Inter-layer prediction method for multi-layer video and device therefor

ABSTRACT

An inter-layer prediction method and device for a multi-layer video. The inter-layer prediction method includes the steps of: generating a restored picture in a reference layer through the predictive coding of the reference layer, determining when there is a restored picture in the reference layer for the time corresponding to a current picture in an enhancement layer for coding or decoding of the current picture in the enhancement layer, and performing inter-layer prediction by referencing a picture in the enhancement layer or a restored picture in the reference layer, depending on whether there is restored picture from the same view in the reference layer.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims the benefit under 35 USC 119(a) of Korean PatentApplication Nos. 10-2012-0111928 filed on Oct. 9, 2012, in the KoreanIntellectual Property Office, the entire disclosure of which isincorporated herein by reference for all purposes.

TECHNICAL FIELD

The present invention generally relates to encoding/decoding of videosand, more particularly, to an inter-layer prediction method andapparatus for a multi-layer video.

BACKGROUND ART

Recently, with the very fast development of wireless networks and theInternet, various multimedia services have been activated. Inparticular, with the advent of broadcasting communication convergencenetworks, a standardization task for providing scalability of videoencoding has been conducted to guarantee Quality of Service (QoS) undervarious conditions of an environment in which multimedia is created,transmitted, and consumed, unlike the past in which only compressioncoding technology was developed.

Scalable Video Coding (SVC) technology is configured to adaptivelyreconstruct images having different resolutions (spatial), qualities,and frame rates (temporal) from a single compressed bitstream inconformity with various terminals and network environments. SVC is avideo codec to which hierarchy is assigned to adaptively cope withvarious types of multimedia devices at high compressibility ofH.264/Advanced Video Coding (AVC), and Joint Video Team (JVT) hascarried out standardization of SVC as an amendment of H.264/MPEG-4 PART10.

Further, High Efficiency Video Coding (HEVC) has been standardized as anext-generation video compression standard technology regarded as havingcompression efficiency that is more than twice the compressionefficiency of the conventional H.264/AVC.

HEVC defines a Coding Unit (CU) having a quadtree structure, aPrediction Unit (PU), and a Transform Unit (TU), and uses an additionalin-loop filter such as a Sample Adaptive Offset (SAO) or deblockingfilter. Further, existing intra prediction and inter prediction areimproved, thus enhancing compression coding efficiency.

Meanwhile, as an extension version of the recently standardized HEVC,SVC has also been standardized.

Conventional SVC technology is problematic in that, in inter-layerprediction, when an enhancement layer having temporal and spatialresolution different from that of a reference layer is encoded, apicture in the enhancement layer, for which a picture in the referencelayer cannot be referred to, is present because a reconstructed picturein the reference layer is not present at the same temporal position asthat of a picture in the enhancement layer to be encoded.

DISCLOSURE Technical Problem

An object of the present invention to solve the above-described problemsis to provide a method for performing inter-layer prediction byadaptively determining a reference picture upon encoding or decoding amulti-layer video.

Another object of the present invention to solve the above problems isto provide an apparatus for performing inter-layer prediction byadaptively determining a reference picture upon encoding or decoding amulti-layer video.

Technical Solution

In accordance with an aspect of the present invention to accomplish theabove objects, there is provided an inter-layer prediction method for amulti-layer video, including generating a reconstructed picture in areference layer via predictive coding of the reference layer,determining whether a reconstructed picture in the reference layer ispresent at a time corresponding to a current picture in an enhancementlayer, so as to encode or decode the current picture in the enhancementlayer, and performing inter-layer prediction with reference to a picturein the enhancement layer or the reconstructed picture in the referencelayer.

Performing the inter-layer prediction may include, when thereconstructed picture in the reference layer is not present at the timecorresponding to the current picture in the enhancement layer,performing inter-layer prediction with reference to a previous orsubsequent picture of the current picture in the enhancement layer.

Performing the inter-layer prediction may include, when thereconstructed picture in the reference layer is not present at the timecorresponding to the current picture in the enhancement layer,performing inter-layer prediction with reference to a previous orsubsequent picture of the reconstructed picture in the reference layerat the time corresponding to the current picture in the enhancementlayer.

Performing the inter-layer prediction may include, when thereconstructed picture in the reference layer is not present at the timecorresponding to the current picture in the enhancement layer,performing inter-layer prediction with reference to a previous orsubsequent picture of the current picture in the enhancement layer and aprevious or subsequent picture of the reconstructed picture in thereference layer at the time corresponding to the current picture in theenhancement layer.

Performing the inter-layer prediction may include, when thereconstructed picture in the reference layer is present at the timecorresponding to the current picture in the enhancement layer,performing inter-layer prediction with reference to the reconstructedpicture in the reference layer at the time corresponding to the currentpicture in the enhancement layer.

In accordance with another aspect of the present invention to accomplishthe above objects, there is provided an inter-layer prediction apparatusfor a multi-layer video, including a frame buffer for storing areconstructed picture in a reference layer, generated via predictivecoding of the reference layer, and a prediction unit for performinginter-layer prediction with reference to a picture in the enhancementlayer or the reconstructed picture in the reference layer so as toencode or decode a current picture in the enhancement layer.

The inter-layer prediction apparatus may further include aninterpolation unit for interpolating the reconstructed picture in thereference layer in conformity with resolution of the enhancement layer.

Advantageous Effects

When the inter-layer prediction method and apparatus for a multi-layervideo according to embodiments of the present invention are used, apicture in an enhancement layer or a picture in a reference layer isadaptively used even when a reconstructed picture in the reference layeris not present at the same temporal position as that of the enhancementlayer, thus performing effective inter-layer prediction.

Further, by means of inter-layer prediction in which a picture in anenhancement layer or a picture in a reference layer is adaptively used,encoding and decoding efficiency may be enhanced.

DESCRIPTION OF DRAWINGS

FIG. 1 is a block diagram showing an encoding apparatus for performinginter-layer prediction for a multi-layer video according to anembodiment of the present invention;

FIG. 2 is a conceptual diagram showing an inter-layer prediction methodfor a multi-layer video according to an embodiment of the presentinvention;

FIG. 3 is a diagram showing a reference picture list based on theinter-layer prediction method for a multi-layer video according to anembodiment of the present invention;

FIG. 4 is a conceptual diagram showing an inter-layer prediction methodfor a multi-layer video according to another embodiment of the presentinvention;

FIG. 5 is a diagram showing a reference picture list based on theinter-layer prediction method for a multi-layer video according toanother embodiment of the present invention;

FIG. 6 is a conceptual diagram showing an inter-layer prediction methodfor a multi-layer video according to a further embodiment of the presentinvention;

FIG. 7 is a diagram showing a reference picture list based on theinter-layer prediction method for a multi-layer video according to afurther embodiment of the present invention;

FIG. 8 is a flowchart showing an inter-layer prediction method for amulti-layer video according to an embodiment of the present invention;

FIG. 9 is diagram illustrating syntax elements used in inter-layerprediction for a multi-layer video according to an embodiment of thepresent invention; and

FIG. 10 is a block diagram showing a decoding apparatus for performinginter-layer prediction for a multi-layer video according to anembodiment of the present invention.

BEST MODE

The present invention may be variously changed and may have variousembodiments, and specific embodiments will be described in detail belowwith reference to the attached drawings. However, it should beunderstood that those embodiments are not intended to limit the presentinvention to specific disclosure forms and they include all changes,equivalents or substitutions included in the spirit and scope of thepresent invention. In the following description of the drawings, thesame reference numerals are used to designate the same or similarelements.

The terms such as “first”, “second”, “A”, and “B” may be used todescribe various components, but those components should not be limitedby the terms. The terms are merely used to distinguish one componentfrom other components. For example, a first component may be designatedas a second component and a second component may be designated as afirst component in the similar manner, without departing from the scopebased on the scope of the present invention. The term “and/or” includesa combination of multiple relevant described items or any of multiplerelevant described items.

Throughout the entire specification, it should be understood that arepresentation indicating that a first component is “connected” or“coupled” to a second component may include the case where the firstcomponent is connected or coupled to the second component with someother component interposed therebetween, as well as the case where thefirst component is directly connected or coupled to the secondcomponent. In contrast, it should be understood that a representationindicating that a first component is “directly connected” or “directlycoupled” to a second component means that no component is interposedbetween the first and second components.

The terms used in the present specification are merely used to describespecific embodiments and are not intended to limit the presentinvention. A singular expression includes a plural expression unless adescription to the contrary is specifically pointed out in context. Inthe present specification, it should be understood that the terms suchas “include” or “have” are merely intended to indicate that features,numbers, steps, operations, components, parts, or combinations thereofare present, and are not intended to exclude a possibility that one ormore other features, numbers, steps, operations, components, parts, orcombinations thereof will be present or added.

Unless differently defined, all terms used here including technical orscientific terms have the same meanings as the terms generallyunderstood by those skilled in the art to which the present inventionpertains. The terms identical to those defined in generally useddictionaries should be interpreted as having meanings identical tocontextual meanings of the related art, and are not interpreted as beingideal or excessively formal meanings unless they are definitely definedin the present specification.

Hereinafter, a video encoding apparatus and a video decoding apparatus,which will be described later, may be any of a user terminal such as aPersonal Computer (PC), a notebook computer, a personal digitalassistant (PDA), a Portable Multimedia Player (PMP), a PlayStationPortable (PSP), a wireless communication terminal, a smart phone orvarious similar types of devices, a server terminal such as a TVapplication server or a service server, or any of various types ofdevices including a communication device, for example, a communicationmodem for performing communication over a wired/wireless communicationnetwork, memory for storing various types of programs and data requiredto encode or decode an image or to perform inter-image or intra-imageprediction for encoding or decoding, a microprocessor for executingprograms and performing operations and control, etc.

Further, an image encoded in a bit stream by the video encodingapparatus is transmitted in real time or in non-real time to the videodecoding apparatus over a wired/wireless communication network, such asthe Internet, a short-range wireless communication network, a WirelessLocal Area Network (WLAN), a Wibro network, or a mobile communicationnetwork, or through various communication interfaces such as a cable ora Universal Serial Bus (USB), so that the encoded image may be decodedby the video decoding apparatus and may be reconstructed into and playedas an original image.

Typically, a video may be composed of a series of pictures, and eachpicture may be divided into predetermined regions such as frames orblocks. When an area of a picture is divided into blocks, the dividedblocks may be chiefly classified into an intra block and an inter blockaccording to a coding method. An intra block denotes a block encodedusing intra prediction coding. Here, intra prediction coding denotes ascheme in which a predicted block is generated by predicting pixels in acurrent block using pixels in blocks, which have been previously encodedand decoded and then reconstructed, in a current picture that iscurrently encoded, and in which differences between pixel values in thepredicted block and in the current block are encoded. Further, an interblock denotes a block encoded using inter prediction coding. Here, interprediction coding denotes a scheme in which a predicted block isgenerated by predicting a current block in a current picture withreference to one or more previous or subsequent pictures, and in whichdifferences between pixel values in the predicted block and the currentblock are encoded. In this case, a frame referred to upon encoding ordecoding the current picture is called a reference frame. Further, thoseskilled in the art to which the present embodiment pertains willappreciate that the term “picture” described below may be replaced withand used with another term, such as an image or a frame, having the samemeaning as the picture. Furthermore, those skilled in the art to whichthe present embodiment pertains will appreciate that a picture referredto in the present invention means a reconstructed picture.

Hereinafter, preferred embodiments of the present invention will bedescribed in detail with reference to the attached drawings.

FIG. 1 is a block diagram showing an encoding apparatus for performinginter-layer prediction for a multi-layer video according to anembodiment of the present invention.

Referring to FIG. 1, the apparatus for performing inter-layer predictionfor a multi-layer video according to the embodiment of the presentinvention may include an encoding apparatus 100 for a reference layerand an encoding apparatus 200 for an enhancement layer. Further, theinter-layer prediction apparatus may include a multiplexer 320 and aninterpolation unit 310.

Each encoding apparatus 100 or 200 includes a subtraction unit 110 or210, a transform unit 120 or 220, a quantization unit 130 or 230, aninverse quantization unit 131 or 231, an inverse transform unit 121 or221, an entropy encoding unit 140 or 240, an addition unit 150 or 250,an in-loop filter unit 160 or 260, a frame memory 170 or 270, anintra-prediction unit 180 or 280, and a motion compensation unit 190 or290.

The subtraction unit 110 or 210 subtracts a predicted image generatedvia intra prediction or inter prediction from an encoding target image(current image) that is a provided input image, and then generates aresidual image between the current image and the predicted image.

The transform unit 120 or 220 functions to transform the residual imagegenerated by the subtraction unit 110 or 210 from a spatial domain intoa frequency domain. Here, the transform unit 120 or 220 may transformthe residual image into a frequency domain using a technique, such as aHadamard transform, a discrete cosine transform, or a discrete sinetransform, for transforming an image signal on a spatial axis into afrequency axis.

The quantization unit 130 or 230 performs quantization on thetransformed data (frequency coefficients) provided by the transform unit120 or 220. That is, the quantization unit 130 or 230 approximates thefrequency coefficients that are pieces of data transformed by thetransform unit 120 or 220 by dividing the frequency coefficients by aquantization step-size, thus calculating quantization result values.

The entropy encoding unit 140 or 240 generates a bit stream byentropy-encoding the quantization result values calculated by thequantization unit 130 or 230. Further, the entropy encoding unit 140 or240 may entropy-encode the quantization result values calculated by thequantization unit 130 or 230 using a Context-Adaptive Variable LengthCoding (CAVLC) or Context-Adaptive Binary Arithmetic Coding (CABAC)technique, and may also entropy-encode information required to decode animage, in addition to the quantization result values.

The inverse quantization unit 131 or 231 inverse-quantizes thequantization result values calculated by the quantization unit 130 or230. That is, the inverse quantization unit 131 or 231 reconstructsvalues in the frequency domain (frequency coefficients) from thequantization result values.

The inverse transform unit 121 or 221 reconstructs the residual image bytransforming the frequency domain values (frequency coefficients)provided by the inverse quantization unit 131 or 231 from the frequencydomain into the spatial domain. The addition unit 150 or 250 generates areconstructed image of the input image by adding the residual image,reconstructed by the inverse transform unit 121 or 221, to the predictedimage generated via intra prediction or inter prediction, and thenstores the reconstructed image in the frame memory 170 or 270.

The frame memory 170 may store a reconstructed picture in the referencelayer, which is generated via predictive coding of the reference layer.Further, the frame memory 270 may store a reconstructed picture in theenhancement layer, which is generated via predictive coding of theenhancement layer, and may store a reconstructed picture in thereference layer or an interpolated picture of the reconstructed picturein the reference layer.

The prediction unit may include the intra-prediction unit 180 or 280 andthe motion compensation unit 190 or 290 and then perform intraprediction and inter prediction.

The intra prediction unit 180 or 280 performs intra prediction, and themotion compensation unit 190 or 290 compensates for motion vectors forinter prediction.

The prediction unit according to the embodiment of the present inventionmay perform inter-layer prediction with reference to a picture in theenhancement layer or a reconstructed picture in the reference layer toencode the current picture in the enhancement layer. Here, inter-layerprediction may mean that intra prediction or inter prediction isperformed with reference to the reconstructed picture in the referencelayer, upon encoding or decoding the enhancement layer.

First, when there is a reconstructed picture in the reference layer atthe temporal position corresponding to the current picture in theenhancement layer, the prediction unit may perform inter-layerprediction with reference to the reconstructed picture in the referencelayer at the time (temporal position) corresponding to the currentpicture in the enhancement layer.

In particular, when the reconstructed picture in the reference layer atthe time corresponding to the current picture in the enhancement layeris not present, the prediction unit may perform inter-layer predictionwith reference to a previous or subsequent picture of the currentpicture in the enhancement layer.

Further, when the reconstructed picture in the reference layer at thetime corresponding to the current picture in the enhancement layer isnot present, the prediction unit may perform inter-layer prediction withreference to a previous or subsequent picture of the reconstructedpicture in the reference layer at the time corresponding to the currentpicture in the enhancement layer.

Furthermore, when the reconstructed picture in the reference layer atthe time corresponding to the current picture in the enhancement layeris not present, the prediction unit may perform inter-layer predictionwith reference to a previous or subsequent picture of the currentpicture in the enhancement layer and a previous or subsequent picture ofthe reconstructed picture in the reference layer at the timecorresponding to the current picture in the enhancement layer.

The in-loop filter unit 160 or 260, which performs filtering on thereconstructed image, may include a Deblocking Filter (DF), a SampleAdaptive Offset (SAO) filter, and an Adaptive Loop Filter (ALF).

The interpolation unit 310 interpolates the reconstructed picture in thereference layer. That is, the interpolation unit 310 may interpolate thereconstructed picture in the reference layer in conformity with thespatial resolution of the enhancement layer. The multiplexer 320receives encoded bit streams from the reference layer and theenhancement layer, and outputs the encoded bit streams.

FIG. 2 is a conceptual diagram showing the inter-layer prediction methodfor a multi-layer video according to an embodiment of the presentinvention.

Referring to FIG. 2, pictures in an enhancement layer may be representedby I_(E0) 20, I_(E1) 21, I_(E2) 22, I_(E3) 23, I_(E4) 24, etc. Further,reconstructed pictures in the enhancement layer may be represented byI_(B0) 25, I_(B2) 26, I_(B4) 27, etc.

At the times corresponding to the pictures in the enhancement layer,that is, I_(E0) 20, I_(E2) 22, and I_(E4) 24, reconstructed pictures inthe reference layer, that is, I_(B0) 25, I_(B2) 26, and I_(B4) 27, arepresent. However, reconstructed pictures in the reference layer may notbe present at the times corresponding to the pictures in the enhancementlayer, that is, I_(E1) 21 and I_(E3) 23.

That is, when a reconstructed picture in the reference layer is presentat the time corresponding to the current picture of the enhancementlayer, the current picture in the enhancement layer may be encoded ordecoded with reference to the corresponding reconstructed picture in thereference layer. Here, the current picture may denote a target pictureto be encoded or decoded.

For example, when a current picture in the enhancement layer is I_(E0)21, I_(E2) 22 or I_(E4) 24, the current picture I_(E0) 21, I_(E2) 22 orI_(E4) 24 may be encoded or decoded with reference to the reconstructedpicture in the reference layer, that is, I_(B0) 25, I_(B2) 26, or I_(B4)27, respectively.

However, when the reconstructed picture in the reference layer at thetime corresponding to the current picture in the enhancement layer isnot present, the current picture in the enhancement layer may be encodedor decoded with reference to a previous or subsequent picture of thecurrent picture in the enhancement layer.

For example, when the current layer in the enhancement layer is I_(E1)21, the current picture I_(E1) 21 may be encoded or decoded withreference to I_(E0) 20 or I_(E2) 22.

Further, when the current picture in the enhancement layer is I_(E3) 23,the current picture I_(E3) 23 may be encoded or decoded with referenceto I_(E2) 22 or I_(E4) 24.

That is, in accordance with an embodiment of the present invention, acurrent picture in the enhancement layer may be encoded or decoded withreference to a picture in the same layer (enhancement layer) to supporttemporal scalability.

FIG. 3 is a diagram showing a reference picture list based on theinter-layer prediction method for a multi-layer video according to anembodiment of the present invention.

Referring to FIG. 3, when a reconstructed picture in the reference layerat the time corresponding to the current picture in the enhancementlayer is not present, a reference list for the current picture in theenhancement layer may include a previous or subsequent picture of thecurrent picture in the enhancement layer.

For example, when the current picture in the enhancement layer is I_(E1)21, picture I_(E0) 20 or I_(E2) 22 may be included in the referencelist.

FIG. 4 is a conceptual diagram showing an inter-layer prediction methodfor a multi-layer video according to another embodiment of the presentinvention.

Referring to FIG. 4, pictures in an enhancement layer may be representedby I_(E0) 40, I_(E1) 41, I_(E2) 42, I_(E3) 43, I_(E4) 44, etc. Further,reconstructed pictures in the enhancement layer may be represented byI_(B0) 45, I_(B2) 46, I_(B4) 47, etc.

Reconstructed pictures in the reference layer, that is, I_(B0) 45,I_(B2) 46, and I_(B4) 47, are present at the times corresponding topictures in the enhancement layer, that is, I_(E0) 40, I_(E2) 42, andI_(E4) 44. However, reconstructed pictures in the reference layer maynot be present at the times corresponding to pictures in the enhancementlayer, that is, I_(E1) 41 and I_(E3) 43.

That is, when a reconstructed picture in the reference layer at the timecorresponding to the current picture in the enhancement layer ispresent, the current picture in the enhancement layer may be encoded ordecoded with reference to the corresponding reconstructed picture in thereference layer.

For example, when the current picture in the enhancement layer is I_(E0)40, I_(E2) 42 or I_(E4) 44, the current picture I_(E0) 40, I_(E2) 42 orI_(E4) 44 may be encoded or decoded with reference to the reconstructedpicture in the reference layer, that is, I_(B0) 45, I_(B2) 46 or I_(B4)47, respectively.

However, when the reconstructed picture in the reference layer at thetime corresponding to the current picture in the enhancement layer isnot present, the current picture may be encoded or decoded withreference to a previous or subsequent picture of the reconstructedpicture in the reference layer at the time corresponding to the currentpicture in the enhancement layer.

For example, when the current picture in the enhancement layer is I_(E1)41, the current picture I_(E1) 41 may be encoded or decoded withreference to picture I_(B0) 45 or I_(B2) 46.

Also, when the current picture in the enhancement layer is I_(E3) 43,the current picture I_(E3) 43 may be encoded or decoded with referenceto picture I_(B2) 46 or I_(B4) 47.

That is, in accordance with an embodiment of the present invention, thecurrent picture in the enhancement layer may be encoded or decoded withreference to a previous or subsequent picture of the reconstructedpicture in the reference layer at the time corresponding to the currentpicture in the enhancement layer to support temporal scalability.

FIG. 5 is a diagram showing a reference picture list based on theinter-layer prediction method for a multi-layer video according toanother embodiment of the present invention.

Referring to FIG. 5, when a reconstructed picture in the reference layerat the time corresponding to the current picture in the enhancementlayer is not present, the reference list for the current picture in theenhancement layer may include a previous or subsequent picture of thereconstructed picture in the reference layer at the time correspondingto the current picture in the enhancement layer.

For example, when the current picture in the enhancement layer is I_(E1)41, picture I_(B0) 45 or I_(B2) 46 may be included in the referencelist.

FIG. 6 is a conceptual diagram showing an inter-layer prediction methodfor a multi-layer video according to a further embodiment of the presentinvention.

Referring to FIG. 6, pictures in an enhancement layer may be representedby I_(E0) 60, I_(E1) 61, I_(E2) 62, I_(E3) 63, I_(E4) 64, etc. Further,reconstructed pictures in the enhancement layer may be represented byI_(B0) 65, I_(B2) 66, I_(B4) 67, etc.

Reconstructed pictures in the reference layer, that is, I_(B0) 65,I_(B2) 66, and I_(B4) 67, are present at the times corresponding to thepictures in the enhancement layer, that is, I_(E0) 60, I_(E2) 62, andI_(E4) 64. However, reconstructed pictures in the reference layer maynot be present at the times corresponding to the pictures in theenhancement layer, that is, I_(E1) 61, and I_(E3) 63.

That is, when a reconstructed picture in the reference layer at the timecorresponding to the current picture in the enhancement layer ispresent, the current picture in the enhancement layer may be encoded ordecoded with reference to the reconstructed picture in the referencelayer.

For example, when the current picture in the enhancement layer is I_(E0)60, I_(E2) 62, or I_(E4) 64, the current picture I_(E0) 60, I_(E2) 62 orI_(E4) 64 may be encoded or decoded with reference to the reconstructedpicture in the reference layer, that is, I_(B0) 65, I_(B2) 66, or I_(B4)67, respectively.

However, when a reconstructed picture in the reference layer at the timecorresponding to the current picture in the enhancement layer is notpresent, the current picture may be encoded or decoded with reference toa previous or subsequent picture of the current picture in theenhancement layer and a previous or subsequent picture of thereconstructed picture in the reference layer at the time correspondingto the current picture in the enhancement layer.

For example, when the current picture in the enhancement layer is I_(E1)61, I_(E1) 61 may be encoded or decoded with reference to picturesI_(E0) 60 or IE₂ 62 and I_(B0) 65 or I_(B2) 66.

Further, when the current picture in the enhancement layer is I_(E3) 63,I_(E3) 63 may be encoded or decoded with reference to pictures I_(E2) 62or I_(E4) 64 and I_(B2) 66 or I_(B4) 67.

That is, according to an embodiment of the present invention, a currentpicture in the enhancement layer may be encoded or decoded withreference to a previous or subsequent picture of the current picture inthe enhancement layer and a previous or subsequent picture of thereconstructed picture in the reference layer at the time correspondingto the current picture in the enhancement layer so as to supporttemporal scalability.

FIG. 7 is a diagram showing a reference picture list based on theinter-layer prediction method for a multi-layer video according to afurther embodiment of the present invention.

Referring to FIG. 7, when a reconstructed picture in a reference layerat the time corresponding to a current picture in an enhancement layeris not present, the reference picture list may include a previous orsubsequent picture of the current picture in the enhancement layer and aprevious or subsequent picture of the reconstructed picture in thereference layer at the time corresponding to the current picture in theenhancement layer.

For example, when the current picture in the enhancement layer is I_(E1)41, picture I_(E3) 60 or I_(E2) 62 and picture I_(B0) 65 or I_(B2) 66may be included in the reference list.

FIG. 8 is a flowchart showing an inter-layer prediction method for amulti-layer video according to an embodiment of the present invention.

Referring to FIG. 8, a reconstructed picture in a reference layer isgenerated via predictive coding of the reference layer, and it may bedetermined whether a reconstructed picture in the reference layer ispresent at the time corresponding to a current picture in an enhancementlayer to encode or decode the current picture in the enhancement layer.

Further, inter-layer prediction may be performed with reference to apicture in the enhancement layer or a reconstructed picture in thereference layer.

First, when a reconstructed picture in the reference layer at the timecorresponding to the current picture in the enhancement layer ispresent, inter-layer prediction may be performed on the current picturein the enhancement layer using the reconstructed picture in thereference layer present at the time corresponding to the current picturein the enhancement layer.

Meanwhile, when the reconstructed picture in the reference layer at thetime corresponding to the current picture in the enhancement layer isnot present, inter-layer prediction may be performed with reference to aprevious or subsequent picture of the current picture in the enhancementlayer.

Further, when the reconstructed picture in the reference layer at thetime corresponding to the current picture in the enhancement layer isnot present, inter-layer prediction may be performed with reference to aprevious or subsequent picture of the reconstructed picture in thereference layer at the time corresponding to the current picture in theenhancement layer.

Furthermore, when the reconstructed picture in the reference layer atthe time corresponding to the current picture in the enhancement layeris not present, inter-layer prediction may be performed with referenceto a previous or subsequent picture of the current picture in theenhancement layer and a previous or subsequent picture of thereconstructed picture in the reference layer at the time correspondingto the current picture in the enhancement layer.

For example, it may be determined whether a reconstructed picture in thereference layer is present at the time corresponding to the currentpicture in the enhancement layer (S810). That is, it may be determinedwhether a reconstructed picture in the reference layer is present at thesame temporal position as that of the current picture in the enhancementlayer.

When it is determined that the reconstructed picture in the referencelayer is present at the time corresponding to the current picture in theenhancement layer, inter-layer prediction may be performed using thereconstructed picture in the reference layer at the time correspondingto the current picture in the enhancement layer (S820).

When it is determined that the reconstructed picture in the referencelayer is not present at the time corresponding to the current picture inthe enhancement layer, inter_layer_pred_ext_enable_flag that is a syntaxelement indicating whether to use a picture at the time differing fromthat of the current picture in the enhancement layer as a referencepicture is decoded.

It may be determined whether inter_layer_pred_ext_enable_flag=1 issatisfied (S830). When inter_layer pred_ext_enable_flag=1 is notsatisfied, intra prediction may be performed on the current picture inthe enhancement layer (S840).

When inter layer_pred_ext_enable_flag=1 is satisfied, it may bedetermined which picture is to be used to perform inter-layer predictiondepending on the value of a syntax element inter_layer_pred_ext_mode.

It may be individually determined whether the value of inter_layerpred_ext_mode is 0 or 1 (S850, S870).

When inter layer_pred_ext_mode=0 is satisfied, inter-layer predictionmay be performed with reference to a previous or subsequent picture ofthe current picture in the enhancement layer (S860). That is, intraprediction or inter prediction may be performed using a reconstructedpicture in the same layer.

When inter layer_pred_ext_mode=1 is satisfied, inter-layer predictionmay be performed with reference to a previous or subsequent picture ofthe reconstructed picture in the reference layer at the timecorresponding to the current picture in the enhancement layer (S880).

When inter layer_pred_ext_mode=1 is not satisfied (when inter_layerpred_ext_mode=2), inter-layer prediction may be performed with referenceto a previous or subsequent picture of the current picture in theenhancement layer and a previous or subsequent picture of thereconstructed picture in the reference layer at the time correspondingto the current picture in the enhancement layer (S890).

FIG. 9 is a diagram illustrating syntax elements used in inter-layerprediction for a multi-layer video according to an embodiment of thepresent invention.

Referring to FIG. 9, it is possible to determine whether to use apicture at a different time as a reference picture by using interlayer_pred_ext_enable_flag that is a syntax element according to anembodiment of the present invention, and also determine which picture isto be used as a reference picture by using inter_layer pred_ext_mode.

FIG. 10 is a block diagram showing a decoding apparatus for performinginter-layer prediction for a multi-layer video according to anembodiment of the present invention.

Referring to FIG. 10, a decoding apparatus according to an embodiment ofthe present invention includes a decoding apparatus 400 for a referencelayer and an encoding apparatus 500 for an enhancement layer. Further,the decoding apparatus includes an interpolation unit 610.

Each decoding apparatus 400 or 500 includes an entropy decoding unit 410or 510, an inverse quantization unit 420 or 520, an inverse transformunit 430 or 530, an addition unit 440 or 540, an in-loop filter unit 450or 550, a frame memory 460 or 560, an intra-prediction unit 470 or 570,and a motion compensation unit 480 or 580. Components of the decodingapparatus 400 or 500 may be understood to respectively correspond tothose of the encoding apparatus of FIG. 1, and thus a detaileddescription thereof will be omitted.

The prediction unit may include the intra-prediction unit 470 or 570 andthe motion compensation unit 480 or 580 to perform intra prediction andinter prediction.

The prediction unit of the decoding apparatus according to theembodiment of the present invention may perform inter-layer predictionwith reference to a picture in an enhancement layer or a reconstructedpicture in a reference layer so as to encode a current picture in theenhancement layer.

First, when a reconstructed picture in the reference layer at the timecorresponding to the current picture in the enhancement layer ispresent, the prediction unit may perform inter-layer prediction withreference to the reconstructed picture in the reference layer at thetime corresponding to the current picture in the enhancement layer.

In particular, when the reconstructed picture in the reference layer atthe time corresponding to the current picture in the enhancement layeris not present, the prediction unit may perform inter-layer predictionwith reference to a previous or subsequent picture of the currentpicture in the enhancement layer.

Further, when the reconstructed picture in the reference layer at thetime corresponding to the current picture in the enhancement layer isnot present, the prediction unit may perform inter-layer prediction withreference to a previous or subsequent picture of the reconstructedpicture in the reference layer at the time corresponding to the currentpicture in the enhancement layer.

Furthermore, when the reconstructed picture in the reference layer atthe time corresponding to the current picture in the enhancement layeris not present, the prediction unit may perform inter-layer predictionwith reference to a previous or subsequent picture of the currentpicture in the enhancement layer and a previous or subsequent picture ofthe reconstructed picture in the reference layer at the timecorresponding to the current picture in the enhancement layer.

The inter-layer prediction method and apparatus for a multi-layer videoaccording to the embodiments of the present invention may effectivelyperform inter-layer prediction by adaptively utilizing a picture in anenhancement layer or a picture in a reference layer even when areconstructed picture in the reference layer is not present at the sametemporal position as a current picture upon encoding or decoding theenhancement layer, thus improving encoding efficiency via effectiveinter-layer prediction.

Further, although the individual components of the above-describedencoding apparatus and decoding apparatus according to the embodimentsof the present invention have been described as being separately listedand included as respective components for convenience of description, ascheme may be implemented in which at least two of the respectivecomponents are combined into a single component, or in which a singlecomponent is divided into a plurality of components and functions areperformed by those components alone or in combination. Those embodimentsin which individual components are combined or separated may also beincluded in the scope of the present invention without departing fromthe essential features of the present invention.

Also, the encoding apparatus and the decoding apparatus according to thepresent invention may be stored in a computer-readable storage mediumand may be implemented in a computer-readable program or code. Thecomputer-readable storage medium includes all types of recording devicesin which data readable by a computer system is stored. Further, thecomputer-readable storage medium may be distributed across computersystems connected over a network and the computer-readable program orcode may be stored and executed in a distributed manner.

Although the present invention has been described with reference topreferred embodiments of the present invention, those skilled in the artwill appreciate that the present invention may be changed or modified invarious manners without departing from the spirit and scope of theinvention as disclosed in the accompanying claims.

The invention claimed is:
 1. An inter-layer prediction apparatus for amulti-layer video, comprising: a frame buffer configured to store areconstructed picture in an enhancement layer and a reconstructedpicture in the reference layer; and a processor configured to determinewhether the reconstructed picture in the reference layer is present at atime corresponding to a current picture in the enhancement layer,determine an inter-layer reference picture for the current picture, inresponse to the reconstructed picture in the reference layer beingdetermined to be present at the time corresponding to the currentpicture, generate a reference picture list, for the current picture,including the inter-layer reference picture and the reconstructedpicture in the enhancement layer, and perform inter prediction on thecurrent picture based on the reference picture list.
 2. The inter-layerprediction apparatus of claim 1, wherein the processor is furtherconfigured to interpolate the reconstructed picture in the referencelayer in conformity with resolution of the enhancement layer.
 3. Theinter-layer prediction apparatus of claim 1, wherein the processor isfurther configured to perform the inter prediction with reference to aprevious or subsequent picture of the current picture in the enhancementlayer, in response to the reconstructed picture in the reference layernot being present at the time corresponding to the current picture inthe enhancement layer.
 4. The inter-layer prediction apparatus of claim1, wherein the processor is further configured to perform the interprediction with reference to a previous or subsequent picture of thereconstructed picture in the reference layer at the time correspondingto the current picture in the enhancement layer, in response to thereconstructed picture in the reference layer not being present at thetime corresponding to the current picture in the enhancement layer. 5.The inter-layer prediction apparatus of claim 1, wherein the processoris further configured to perform the inter prediction with reference toa previous or subsequent picture of the current picture in theenhancement layer and a previous or subsequent picture of thereconstructed picture in the reference layer at the time correspondingto the current picture in the enhancement layer, in response to thereconstructed picture in the reference layer not being present at thetime corresponding to the current picture in the enhancement layer. 6.The inter-layer prediction apparatus of claim 1, wherein the processoris further configured to perform the inter prediction with reference tothe reconstructed picture in the reference layer at the timecorresponding to the current picture in the enhancement layer, inresponse to the reconstructed picture in the reference layer beingpresent at the time corresponding to the current picture in theenhancement layer.